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Report  Title 

Final  Report:  The  Empathic  Operating  System  (emOS) 

ABSTRACT 

Physiological  measurements  have  typically  been  limited  to  expensive  and  cumbersome  clinical  research  equipment,  causing  the  relationship 
between  how  stress  and  rest  affect  performance  to  be  poorly  understood.  To  solve  this  gap,  Neumitra  develops  wearable  technologies  with 
continuous  monitoring  of  stress  and  rest  during  daily  life  demands.  Our  mobile  software  incorporates  users’  physiology  with  their  contextual 
data,  including  phone  calls,  locations,  and  events,  to  help  mitigate  the  effects  of  stress  on  daily  life.  This  approach  offers  novel  opportunities 
to  inform  computing  with  personal  biological  needs. 

This  proposal  extends  our  technologies  and  those  of  third  parties  to  apply  personalized  physiologic  computations  with  self-learning  software 
to  enhance  continuous  performance  and  productivity.  The  “Empathic  Operating  System”  fosters  a  symbiosis  between  biology  and 
technology  throughout  daily  life  demands.  The  interactive  relationship  between  physiological  data  and  software  that  we  develop  will 
empower  cognitive  performance  based  on  real-time  human  needs.  This  empathic  software  will  respond  to  how  you  feel  with  insights  and 
recommendations  to  help  us  all  perform  at  our  best  when  we  are  most  limited.  By  developing  this  approach  for  third  party  devices,  we  will 
foster  a  wide  variety  of  integration  capabilities  in  a  mobile  applications  library  to  foment  radically  new  approaches  in  personal  physiologic 
computing. 
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The  Problem  Studied 


Physiological  measurements  have  typically  been  limited  to  expensive  and  cumbersome  clinical  research 
equipment,  causing  the  relationship  between  how  stress  and  rest  affect  performance  to  be  poorly 
understood.  To  solve  this  gap,  Neumitra  develops  wearable  technologies  with  continuous  monitoring  of 
stress  and  rest  during  daily  life  demands.  Our  mobile  software  incorporates  users’  physiology  with  their 
contextual  data,  including  phone  calls,  locations,  and  events,  to  help  mitigate  the  effects  of  stress  on  daily 
life.  This  approach  offers  novel  opportunities  to  inform  computing  with  personal  biological  needs. 

This  proposal  extends  our  technologies  and  those  of  third  parties  to  apply  personalized  physiologic 
computations  with  self-learning  software  to  enhance  continuous  performance  and  productivity.  The 
“Empathic  Operating  System”  fosters  a  symbiosis  between  biology  and  technology  throughout  daily  life 
demands.  The  interactive  relationship  between  physiological  data  and  software  that  we  develop  will 
empower  cognitive  performance  based  on  real-time  human  needs.  This  empathic  software  will  respond  to 
how  you  feel  with  insights  and  recommendations  to  help  us  all  perform  at  our  best  when  we  are  most 
limited.  By  developing  this  approach  for  third  party  devices,  we  will  foster  a  wide  variety  of  integration 
capabilities  in  a  mobile  applications  library  to  foment  radically  new  approaches  in  personal  physiologic 
computing. 

Statement  of  Work 

From  January  1  to  April  7  (‘pre-award  period’)  we  focused  our  efforts  on  putting  together  the  emOS  team 
such  that  building  out  the  emOS  mobile  application  would  begin  immediately  upon  award  of  the  grant. 
We  on-boarded  with  several  recruiters  and  candidate  sourcing  websites,  from  which  we  interviewed  and 
on-boarded  a  Project  Manager,  a  Lead  iOS  Developer,  and  a  TJX  Designer.  We  also  interviewed  over  50 
Android  candidates  and  identified  four  viable  Lead  Android  Developers;  however,  we  put  their  offers  on 
hold  due  to  delay  on  grant  award.  We  have  identified  and  secured  a  backup  Android  development  shop, 
Raiz  Labs,  to  ensure  timelines  and  budgets  are  followed  if  Android  resources  cannot  be  secured.  Raiz 
Labs  is  a  local  collaborative  mobile  application  group  that  does  two-week  sprints,  delivers  builds  every 
one  to  two  weeks,  and  holds  face-to-face  meetings  with  collaborators  multiple  times  a  week. 

We  also  focused  our  efforts  on  optimizing  hardware  options  for  the  emOS  mobile  application.  To  de-risk 
our  own  hardware,  we  validated  Microsoft  Band  2  data  streams,  as  this  device  is  the  only  commercially 
available  wearable  with  the  appropriate  sensors  for  compatibility  with  our  analytics.  We  sourced  and 
began  validation  with  Ineda  loT  and  Wearable  SoC  platform  for  Neumitra’s  in-house  hardware  as  well. 

As  requested  by  ARO,  during  the  pre-award  period  we  have  updated  our  IRB  to  reflect  the  current  grant 
proposal  with  specifications  for  privacy  and  consenting  concerns.  Lastly,  with  the  on-boarded  emOS  team 
members,  we  created  a  design/product  discovery  and  validation  process,  as  well  as  a  roadmap  for  iOS  and 
Android  development  with  privacy  and  data  architecture.  All  activity  during  the  pre-award  was 
documented  to  DARPA  through  regular  emails  or  phone  calls,  as  well  as  in-person  during  our  kick-off 
meeting. 

Work  Schedule  for  Project  Tasks 

January  2016 

The  Project  Manager  for  the  emOS  team  was  hired  on  January  1,  2016,  and  began  work  with  Neumitra  on 
January  4,  2016.  The  Project  Manager  began  an  intensive  search  for  project  staff  while  concurrently 
carrying  out  a  deep-dive  and  knowledge  transfer  for  technologies  and  previous  approaches  and 
implementations.  The  Project  Manager  began  interviewing  Android  and  iOS  developers,  and  began  the 


interview  process  with  the  Lead  iOS  developer  that  was  ultimately  hired  in  February.  As  requested  by  the 
DHRPO,  the  Clinical  Engineer  began  incorporating  the  emOS  grant  into  an  IRB  proposal.  Additionally, 
the  pre-existing  User  Interface  designer  and  Junior  iOS  developer  furthered  development  of  iOS  app 
based  on  a  previous  approach. 

Deliverable:  Project  Manager  hired  and  integrated  with  existing  efforts 


February  2016 

February  started  with  an  informal  kickoff  meeting  between  the  DARPA  team  and  Neumitra  employees. 
The  Lead  iOS  developer  was  hired  on  February  10,  2016.  Following  this,  the  part-time  User  Experience 
designer  was  hired  on  February  15,  2016. 

Deliverable:  Lead  iOS  Developer  and  part-time  UX  designer  hired 


March  2016 

The  Lead  iOS  developer  and  User  Experience  designer  began  at  Neumitra,  and  the  Project  Manager  held 
an  internal  kickoff  with  the  emOS  team  -  the  Project  Manager,  User  Interface  designer,  User  Experience 
designer,  and  iOS  developers.  The  emOS  team  began  the  design  process  for  iOS  and  Android  joint  design 
and  development.  Additionally,  knowledge  transfer  between  the  User  Interface  designer  and  User 
Experience  designer  began,  which  consisted  of  a  review  of  three  years  of  designs  and  rationale.  The 
emOS  team  investigated  Spotify  and  IFTTT  APIs  and  reaffirmed  the  expected  level  of  effort.  The  team 
created  a  Microsoft  Band  2  data  pipeline  for  iOS,  providing  a  third  party  ecosystem  to  de-risk  our  own 
hardware.  After  carrying  out  a  User  Interface/User  Experience  brainstorming  session  with  the  emOS 
team,  the  team  began  engagement  personas  research  for  design  for  iOS  and  Android.  User  Experience 
concepts  were  abstracted  and  tested,  with  anecdotal  evidence  providing  a  cardinal  direction  for  app 
development. 

Deliverable:  Lead  iOS  Developer  and  part-time  UX  designer  integrated  with  existing  and  new  efforts 

Deliverable:  Designed  modular  and  proper  iOS  architecture  and  design  and  tested  third  party  integration 
strategy  (Spotify,  Facebook,  Box,  Nest,  Dropbox),  app  database  (Realm),  and  location  sendees 


April  2016 

April  began  with  significant  movement  on  Android  pipeline,  and  we  began  to  see  highly  qualified 
candidates.  By  the  first  week  of  April,  paper  prototypes  for  a  new  User  Interface  for  iOS  were  complete. 
The  User  Interface/User  Experience  survey,  developed  in  March,  was  fielded  with  over  30  responses  and 
3  in-person  interviews.  We  received  a  ‘Stop  Work’  notice  on  April  7th. 

Deliverable:  Paper  prototypes  of  new  UI  framework  for  completed 

Deliverable:  UXJUl  survey  fielded  and  analyzed 


